1 Example hpgltool usage with a real data set (fission)
This document aims to provide further examples in how to use the hpgltools.
Note to self, the header has rmarkdown::pdf_document instead of html_document or html_vignette because it gets some bullcrap error ‘margins too large’…
1.1 Setting up
Here are the commands I invoke to get ready to play with new data, including everything required to install hpgltools, the software it uses, and the fission data.
if (! "BiocInstaller" %in% installed.packages()) {
source("http://bioconductor.org/biocLite.R")
}
if (! "devtools" %in% installed.packages()) {
biocLite("devtools")
}
if (! "hpgltools" %in% installed.packages()) {
devtools::install_github("elsayed-lab/hpgltools")
}
if (! "fission" %in% installed.packages()) {
biocLite("fission")
}
## I use the function 'sm()' to quiet loud functions.
tt <- sm(library(fission))
tt <- sm(data(fission))1.2 Data import
All the work I do in Dr. El-Sayed’s lab makes some pretty hard assumptions about how data is stored. As a result, to use the fission data set I will do a little bit of shenanigans to match it to the expected format. Now that I have played a little with fission, I think its format is quite nice and am likely to have my experiment class instead be a SummarizedExperiment.
## Extract the meta data from the fission dataset
meta <- as.data.frame(fission@colData)
## Make conditions and batches
meta$condition <- paste(meta$strain, meta$minute, sep=".")
meta$batch <- meta$replicate
meta$sample.id <- rownames(meta)
## Grab the count data
fission_data <- fission@assays$data$counts
## This will make an experiment superclass called 'expt' and it contains
## an ExpressionSet along with any arbitrary additional information one might want to include.
## Along the way it writes a Rdata file which is by default called 'expt.Rdata'
fission_expt <- create_expt(metadata=meta, count_dataframe=fission_data)## Reading the sample metadata.## The sample definitions comprises: 36, 7 rows, columns.## Matched 7039 annotations and counts.## Bringing together the count matrix and gene information.1.3 Normalizing and exploring data
There are lots of toys we have learned to use to play with with raw data and explore stuff like batch effects or non-canonical distributions or skewed counts. hpgltools provides some functionality to make this process easier. The graphs shown below and many more are generated with the wrapper ‘graph_metrics()’ but that takes away the chance to explain the graphs as I generate them.
## First make a bar plot of the library sizes in the experiment.
## Notice that the colors were auto-chosen by create_expt() and they should
## be maintained throughout this process
fis_libsize <- plot_libsize(fission_expt)
fis_libsize$plot
## Here we see that the wild type replicate 3 sample for 15 minutes has fewer non-zero genes than all its friends.
fis_nonzero <- plot_nonzero(fission_expt, labels="boring", title="nonzero vs. cpm")
fis_nonzero$plot
fis_density <- plot_density(fission_expt)## This data will benefit from being displayed on the log scale.## If this is not desired, set scale='raw'## Some entries are 0. We are on log scale, setting them to 0.5.## Changed 24130 zero count features.fis_density$plot
fis_density$condition_summary## condition min 1st median mean 3rd max
## 1: wt.0 0.5 31 216 1839 647 5346309
## 2: wt.15 0.5 20 135 1819 474 6733064
## 3: wt.30 0.5 27 183 1917 559 6722419
## 4: wt.60 0.5 27 195 1676 627 5034452
## 5: wt.120 0.5 25 183 1540 529 4531663
## 6: wt.180 0.5 29 210 1904 600 7223993
## 7: mut.0 0.5 35 238 1826 716 4818286
## 8: mut.15 0.5 22 142 1582 461 4387929
## 9: mut.30 0.5 33 226 2028 684 6061779
## 10: mut.60 0.5 39 279 1981 772 5268796
## 11: mut.120 0.5 35 242 1763 675 4174104
## 12: mut.180 0.5 24 178 1534 510 4276682fis_density$batch_summary## batch min 1st median mean 3rd max
## 1: r1 0.5 30 205 1908 632 6733064
## 2: r2 0.5 28 193 1727 586 7223993
## 3: r3 0.5 27 196 1717 600 6722419fis_density$sample_summary## sample min 1st median mean 3rd max
## 1: GSM1368273 0.5 46.0 306 2225.5 869.0 4542884
## 2: GSM1368274 0.5 21.0 147 1344.9 411.0 4117160
## 3: GSM1368275 0.5 34.0 243 1946.5 678.0 5346309
## 4: GSM1368276 0.5 34.0 226 2625.1 751.5 6733064
## 5: GSM1368277 0.5 20.0 123 1470.8 393.0 4094540
## 6: GSM1368278 0.5 13.0 95 1360.3 312.0 4714248
## 7: GSM1368279 0.5 38.0 257 2321.0 754.5 5624485
## 8: GSM1368280 0.5 19.0 138 1528.6 406.0 4966659
## 9: GSM1368281 0.5 26.0 182 1902.2 517.0 6722419
## 10: GSM1368282 0.5 15.0 106 996.5 301.0 3160098
## 11: GSM1368283 0.5 50.0 357 2451.7 1009.5 5034452
## 12: GSM1368284 0.5 31.0 229 1578.8 636.0 3781322
## 13: GSM1368285 0.5 33.0 238 1784.7 665.0 3855298
## 14: GSM1368286 0.5 23.0 170 1386.8 481.5 3419088
## 15: GSM1368287 0.5 21.0 161 1447.8 452.0 4531663
## 16: GSM1368288 0.5 32.0 224 1853.6 614.5 4562092
## 17: GSM1368289 0.5 35.0 252 2351.9 704.0 7223993
## 18: GSM1368290 0.5 22.0 172 1505.2 473.0 4711077
## 19: GSM1368291 0.5 23.0 154 1325.8 439.0 3317558
## 20: GSM1368292 0.5 39.0 267 1821.3 738.5 4093207
## 21: GSM1368293 0.5 52.0 357 2330.7 1009.5 4818286
## 22: GSM1368294 0.5 19.0 120 1473.7 385.0 3656215
## 23: GSM1368295 0.5 25.5 168 1713.5 521.5 4387929
## 24: GSM1368296 0.5 22.0 147 1557.8 469.0 4337853
## 25: GSM1368297 0.5 37.0 255 2580.2 775.5 6061779
## 26: GSM1368298 0.5 28.0 189 1721.2 580.0 4262642
## 27: GSM1368299 0.5 35.0 239 1781.4 713.0 3280103
## 28: GSM1368300 0.5 41.0 294 2026.0 824.5 3590986
## 29: GSM1368301 0.5 36.0 248 1845.1 674.0 5268796
## 30: GSM1368302 0.5 39.0 302 2072.8 833.5 5229526
## 31: GSM1368303 0.5 36.0 250 1884.7 699.5 3935733
## 32: GSM1368304 0.5 33.0 224 1670.9 612.0 4174104
## 33: GSM1368305 0.5 36.0 260 1733.5 725.0 3387186
## 34: GSM1368306 0.5 29.0 203 1798.8 582.5 4276682
## 35: GSM1368307 0.5 26.0 184 1416.2 513.5 3266544
## 36: GSM1368308 0.5 19.0 153 1385.7 433.0 4232664
## sample min 1st median mean 3rd max1.3.1 An initial pca plot
In most cases, raw data does not cluster very well, lets see if that is also true for the fission experiment. Assuming it doesn’t, lets normalize the data using the defaults (cpm, quantile, log2) and try again.
## Something in this is causing a build loop on travis...
## Unsurprisingly, the raw data doesn't cluster well at all...
fis_rawpca <- plot_pca(fission_expt, expt_labels=fission_expt$condition, cis=NULL)
fis_rawpca$plot
## So, normalize the data
norm_expt <- sm(normalize_expt(fission_expt, transform="log2", norm="quant", convert="cpm"))
## And try the pca again
fis_normpca <- plot_pca(norm_expt, plot_labels="normal", title="normalized pca", cis=NULL)
fis_normpca$plot
summary(fis_normpca)## Length Class Mode
## pca 3 -none- list
## plot 9 gg list
## table 8 data.frame list
## res 4 data.frame list
## variance 35 -none- numericIn the final line of the preceeding block, I printed a summary of the return from plot_pca(). It contains the following information:
- pca: The result from the fast.svd() call.
- plot: The ggplot2 pca plot.
- table: The metadata used to make the pca plot.
- res: A table of the residual variance after each component by condition/batch.
- variance: A numeric list of the %variance remaining after each PC.
With that in mind, lets perform some more pca plots after normalizing the data and see how different they look.
normbatch_expt <- sm(normalize_expt(fission_expt, transform="log2", norm="quant",
convert="cpm", batch="sva"))
fis_normbatchpca <- plot_pca(normbatch_expt,
title="Normalized PCA with batch effect correction.", cis=NULL)
fis_normbatchpca$plot
## ok, that caused the 0, 60, 15, and 30 minute samples to cluster nicely
## the 120 and 180 minute samples are still a bit tight
## pca_information provides some more information about the call to
## fast.svd that went into making the pca plot
fis_info <- pca_information(norm_expt,
expt_factors=c("condition","batch","strain","minute"),
num_components=6)## More shallow curves in these plots suggest more genes in this principle component.
## The r^2 table shows that quite a lot of the variance in the data is explained by condition
head(fis_info$rsquared_table)## prop_var cumulative_prop_var condition batch strain minute
## 1 36.642 36.64 99.14 0.017 0.123 98.832
## 2 14.010 50.65 96.67 0.382 0.006 95.833
## 3 8.432 59.08 66.40 25.100 1.834 61.469
## 4 5.807 64.89 38.72 26.589 6.086 29.265
## 5 5.179 70.07 88.88 5.137 0.316 87.439
## 6 3.633 73.70 15.16 0.950 7.738 4.997## We can look at the correlation between the principle components and the factors in the experiment
## in this case looking at condition/batch vs the first 4 components.
fis_info$pca_cor## PC1 PC2 PC3 PC4 PC5 PC6
## condition 0.31602 -0.269379 -0.05825 -0.3246 0.03743 0.28533
## batch 0.01233 -0.060604 0.50079 -0.5105 -0.22602 0.09691
## strain 0.03507 -0.007689 0.13542 -0.2467 0.05617 0.27817
## minute 0.57717 -0.530995 -0.35561 -0.2228 -0.02303 0.08811## And p-values to lend some credence(or not to those assertions)
fis_info$anova_p## PC1 PC2 PC3 PC4 PC5 PC6
## condition 0.0604265 0.1121139 0.735783 0.053393 0.8284 0.09163
## batch 0.9430865 0.7255040 0.001865 0.001468 0.1850 0.57394
## strain 0.8390728 0.9645021 0.431012 0.146923 0.7449 0.10044
## minute 0.0002285 0.0008625 0.033295 0.191462 0.8940 0.60937## Try again with batch removed data
batchnorm_expt <- sm(normalize_expt(fission_expt, batch="limma", norm="quant",
transform="log2", convert="cpm"))
fis_batchnormpca <- plot_pca(batchnorm_expt, plot_title="limma corrected pca", cis=NULL)
fis_batchnormpca$plot
test_pca <- pca_information(batchnorm_expt,
expt_factors=c("condition","batch","strain","minute"),
num_components=6)## More shallow curves in these plots suggest more genes in this principle component.
Interesting, the batch normalized pca plot looks much the same as the normalized. The variances are in fact pretty much the exact same…
1.4 Look at the data distributions
We have some tools which provide visualizations of the distribution of the data:
fission_boxplot <- sm(plot_boxplot(fission_expt))
fission_boxplot
sf_expt <- sm(normalize_expt(fission_expt, norm="sf"))
fission_boxplot <- sm(plot_boxplot(sf_expt))
fission_boxplot
tm_expt <- sm(normalize_expt(fission_expt, norm="tmm"))
fission_boxplot <- sm(plot_boxplot(tm_expt))
fission_boxplot
rle_expt <- sm(normalize_expt(fission_expt, norm="rle"))
fission_boxplot <- sm(plot_boxplot(rle_expt))
fission_boxplot
up_expt <- sm(normalize_expt(fission_expt, norm="upperquartile"))
fission_boxplot <- sm(plot_boxplot(up_expt))
fission_boxplot
fission_density <- plot_density(norm_expt)
fission_density$plot
fission_density <- plot_density(sf_expt)## This data will benefit from being displayed on the log scale.## If this is not desired, set scale='raw'## Some entries are 0. We are on log scale, setting them to 0.5.## Changed 24130 zero count features.fission_density$plot
fission_density <- plot_density(tm_expt)## This data will benefit from being displayed on the log scale.## If this is not desired, set scale='raw'## Some entries are 0. We are on log scale, setting them to 0.5.## Changed 24130 zero count features.fission_density$plot
compare_12 <- plot_single_qq(fission_expt, x=1, y=2)
compare_12$log
1.5 See how they cluster
Ok, so we can further check out how the data cluster with respect to one another…
fission_cor <- plot_corheat(norm_expt)
fission_cor$plot
fission_cor <- plot_corheat(batchnorm_expt)
fission_cor$plot
fission_dis <- plot_disheat(norm_expt)
fission_dis$plot
fission_dis <- plot_disheat(batchnorm_expt)
fission_dis$plot2 variancePartition
A new toy I have been introduced to is variancePartition, let us try that with this data.
test_varpart <- varpart(fission_expt, predictor=NULL, factors=c("condition","batch"))## Attempting mixed linear model with: ~ (1|condition) + (1|batch)## Fitting the expressionset to the model, this is slow.## Projected run time: ~ 0.5 min## Placing factor: condition at the beginning of the model.test_varpart$percent_plot
test_varpart$partition_plot
## Here, let us test the variance contributed by strain, time, and replicate.
test_varpart <- varpart(fission_expt, predictor=NULL, factors=c("condition", "strain", "minute", "replicate"))## Attempting mixed linear model with: ~ (1|condition) + (1|strain) + (1|minute) + (1|replicate)
## Fitting the expressionset to the model, this is slow.## Projected run time: ~ 0.7 min## Placing factor: condition at the beginning of the model.test_varpart$percent_plot
test_varpart$partition_plot
YAY!
pander::pander(sessionInfo())R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
locale: LC_CTYPE=en_US.utf8, LC_NUMERIC=C, LC_TIME=en_US.utf8, LC_COLLATE=en_US.utf8, LC_MONETARY=en_US.utf8, LC_MESSAGES=en_US.utf8, LC_PAPER=en_US.utf8, LC_NAME=C, LC_ADDRESS=C, LC_TELEPHONE=C, LC_MEASUREMENT=en_US.utf8 and LC_IDENTIFICATION=C
attached base packages: parallel, stats4, stats, graphics, grDevices, utils, datasets, methods and base
other attached packages: variancePartition(v.1.10.0), ggplot2(v.2.2.1), fission(v.0.114.0), SummarizedExperiment(v.1.10.1), DelayedArray(v.0.6.0), BiocParallel(v.1.14.1), matrixStats(v.0.53.1), Biobase(v.2.40.0), GenomicRanges(v.1.32.3), GenomeInfoDb(v.1.16.0), IRanges(v.2.14.10), S4Vectors(v.0.18.2), BiocGenerics(v.0.26.0), foreach(v.1.4.4), Vennerable(v.3.1.0.9000), ruv(v.0.9.7) and hpgltools(v.2018.03)
loaded via a namespace (and not attached): minqa(v.1.2.4), Rtsne(v.0.13), colorspace(v.1.3-2), colorRamps(v.2.3), rprojroot(v.1.3-2), htmlTable(v.1.12), corpcor(v.1.6.9), XVector(v.0.20.0), base64enc(v.0.1-3), rstudioapi(v.0.7), ggrepel(v.0.8.0), bit64(v.0.9-7), AnnotationDbi(v.1.42.1), codetools(v.0.2-15), splines(v.3.5.1), doParallel(v.1.0.11), DESeq(v.1.32.0), robustbase(v.0.93-0), geneplotter(v.1.58.0), knitr(v.1.20), ade4(v.1.7-11), Formula(v.1.2-3), nloptr(v.1.0.4), pbkrtest(v.0.4-7), packrat(v.0.4.9-3), annotate(v.1.58.0), cluster(v.2.0.7-1), graph(v.1.58.0), compiler(v.3.5.1), backports(v.1.1.2), Matrix(v.1.2-14), lazyeval(v.0.2.1), limma(v.3.36.1), acepack(v.1.4.1), htmltools(v.0.3.6), tools(v.3.5.1), gtable(v.0.2.0), GenomeInfoDbData(v.1.1.0), reshape2(v.1.4.3), Rcpp(v.0.12.17), gdata(v.2.18.0), preprocessCore(v.1.42.0), nlme(v.3.1-137), iterators(v.1.0.9), stringr(v.1.3.1), lme4(v.1.1-17), openxlsx(v.4.1.0), gtools(v.3.5.0), devtools(v.1.13.5), XML(v.3.98-1.11), edgeR(v.3.22.2), DEoptimR(v.1.0-8), directlabels(v.2018.05.22), zlibbioc(v.1.26.0), MASS(v.7.3-50), scales(v.0.5.0), BiocInstaller(v.1.30.0), doSNOW(v.1.0.16), RBGL(v.1.56.0), RColorBrewer(v.1.1-2), yaml(v.2.1.19), memoise(v.1.1.0), gridExtra(v.2.3), pander(v.0.6.1), rpart(v.4.1-13), latticeExtra(v.0.6-28), stringi(v.1.2.2), RSQLite(v.2.1.1), highr(v.0.6), genefilter(v.1.62.0), checkmate(v.1.8.5), caTools(v.1.17.1), zip(v.1.0.0), rlang(v.0.2.1), bitops(v.1.0-6), evaluate(v.0.10.1), lattice(v.0.20-35), htmlwidgets(v.1.2), labeling(v.0.3), bit(v.1.1-14), plyr(v.1.8.4), magrittr(v.1.5), DESeq2(v.1.20.0), gplots(v.3.0.1), snow(v.0.4-2), Hmisc(v.4.1-1), DBI(v.1.0.0), pillar(v.1.2.3), foreign(v.0.8-70), withr(v.2.1.2), mgcv(v.1.8-24), survival(v.2.42-4), RCurl(v.1.95-4.10), nnet(v.7.3-12), tibble(v.1.4.2), KernSmooth(v.2.23-15), rmarkdown(v.1.9), locfit(v.1.5-9.1), grid(v.3.5.1), sva(v.3.28.0), data.table(v.1.11.4), blob(v.1.1.1), digest(v.0.6.15), xtable(v.1.8-2), genoPlotR(v.0.8.7), munsell(v.0.4.3) and quadprog(v.1.5-5)
LS0tCnRpdGxlOiAiaHBnbHRvb2xzIGV4YW1wbGVzIHVzaW5nIHRoZSBmaXNzaW9uIGRhdGFzZXQiCmF1dGhvcjogImF0YiBhYmVsZXdAZ21haWwuY29tIgpkYXRlOiAiYHIgU3lzLkRhdGUoKWAiCm91dHB1dDoKIGh0bWxfZG9jdW1lbnQ6CiAgY29kZV9kb3dubG9hZDogdHJ1ZQogIGNvZGVfZm9sZGluZzogc2hvdwogIGZpZ19jYXB0aW9uOiB0cnVlCiAgZmlnX2hlaWdodDogNwogIGZpZ193aWR0aDogNwogIGhpZ2hsaWdodDogZGVmYXVsdAogIGtlZXBfbWQ6IGZhbHNlCiAgbW9kZTogc2VsZmNvbnRhaW5lZAogIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogIHNlbGZfY29udGFpbmVkOiB0cnVlCiAgdGhlbWU6IHJlYWRhYmxlCiAgdG9jOiB0cnVlCiAgdG9jX2Zsb2F0OgogICAgY29sbGFwc2VkOiBmYWxzZQogICAgc21vb3RoX3Njcm9sbDogZmFsc2UKdmlnbmV0dGU6ID4KICAlXFZpZ25ldHRlSW5kZXhFbnRyeXtiLTAyX2Zpc3Npb25fZGF0YV9leHBsb3JhdGlvbn0KICAlXFZpZ25ldHRlRW5naW5le2tuaXRyOjpybWFya2Rvd259CiAgXHVzZXBhY2thZ2VbdXRmOF17aW5wdXRlbmN9Ci0tLQoKYGBge3Igb3B0aW9ucywgaW5jbHVkZT1GQUxTRX0KIyMgVGhlc2UgYXJlIHRoZSBvcHRpb25zIEkgdGVuZCB0byBmYXZvcgpsaWJyYXJ5KCJocGdsdG9vbHMiKQprbml0cjo6b3B0c19rbml0JHNldChwcm9ncmVzcz1UUlVFLAogICAgICAgICAgICAgICAgICAgICB2ZXJib3NlPVRSVUUsCiAgICAgICAgICAgICAgICAgICAgIHdpZHRoPTkwLAogICAgICAgICAgICAgICAgICAgICBlY2hvPVRSVUUpCmtuaXRyOjpvcHRzX2NodW5rJHNldChlcnJvcj1UUlVFLAogICAgICAgICAgICAgICAgICAgICAgZmlnLndpZHRoPTgsCiAgICAgICAgICAgICAgICAgICAgICBmaWcuaGVpZ2h0PTgsCiAgICAgICAgICAgICAgICAgICAgICBkcGk9OTYpCm9sZF9vcHRpb25zIDwtIG9wdGlvbnMoZGlnaXRzPTQsCiAgICAgICAgICAgICAgICAgICAgICAgc3RyaW5nc0FzRmFjdG9ycz1GQUxTRSwKICAgICAgICAgICAgICAgICAgICAgICBrbml0ci5kdXBsaWNhdGUubGFiZWw9ImFsbG93IikKZ2dwbG90Mjo6dGhlbWVfc2V0KGdncGxvdDI6OnRoZW1lX2J3KGJhc2Vfc2l6ZT0xMCkpCnNldC5zZWVkKDEpCnJtZF9maWxlIDwtICJiLTAyX2Zpc3Npb25fZGF0YV9leHBsb3JhdGlvbi5SbWQiCmBgYAoKIyBFeGFtcGxlIGhwZ2x0b29sIHVzYWdlIHdpdGggYSByZWFsIGRhdGEgc2V0IChmaXNzaW9uKQoKVGhpcyBkb2N1bWVudCBhaW1zIHRvIHByb3ZpZGUgZnVydGhlciBleGFtcGxlcyBpbiBob3cgdG8gdXNlIHRoZSBocGdsdG9vbHMuCgpOb3RlIHRvIHNlbGYsIHRoZSBoZWFkZXIgaGFzIHJtYXJrZG93bjo6cGRmX2RvY3VtZW50IGluc3RlYWQgb2YgaHRtbF9kb2N1bWVudCBvciBodG1sX3ZpZ25ldHRlCmJlY2F1c2UgaXQgZ2V0cyBzb21lIGJ1bGxjcmFwIGVycm9yICdtYXJnaW5zIHRvbyBsYXJnZScuLi4KCiMjIFNldHRpbmcgdXAKCkhlcmUgYXJlIHRoZSBjb21tYW5kcyBJIGludm9rZSB0byBnZXQgcmVhZHkgdG8gcGxheSB3aXRoIG5ldyBkYXRhLCBpbmNsdWRpbmcgZXZlcnl0aGluZwpyZXF1aXJlZCB0byBpbnN0YWxsIGhwZ2x0b29scywgdGhlIHNvZnR3YXJlIGl0IHVzZXMsIGFuZCB0aGUgZmlzc2lvbiBkYXRhLgoKYGBge3Igc2V0dXB9CmlmICghICJCaW9jSW5zdGFsbGVyIiAlaW4lIGluc3RhbGxlZC5wYWNrYWdlcygpKSB7CiAgc291cmNlKCJodHRwOi8vYmlvY29uZHVjdG9yLm9yZy9iaW9jTGl0ZS5SIikKfQppZiAoISAiZGV2dG9vbHMiICVpbiUgaW5zdGFsbGVkLnBhY2thZ2VzKCkpIHsKICBiaW9jTGl0ZSgiZGV2dG9vbHMiKQp9CmlmICghICJocGdsdG9vbHMiICVpbiUgaW5zdGFsbGVkLnBhY2thZ2VzKCkpIHsKICBkZXZ0b29sczo6aW5zdGFsbF9naXRodWIoImVsc2F5ZWQtbGFiL2hwZ2x0b29scyIpCn0KaWYgKCEgImZpc3Npb24iICVpbiUgaW5zdGFsbGVkLnBhY2thZ2VzKCkpIHsKICBiaW9jTGl0ZSgiZmlzc2lvbiIpCn0KIyMgSSB1c2UgdGhlIGZ1bmN0aW9uICdzbSgpJyB0byBxdWlldCBsb3VkIGZ1bmN0aW9ucy4KdHQgPC0gc20obGlicmFyeShmaXNzaW9uKSkKdHQgPC0gc20oZGF0YShmaXNzaW9uKSkKYGBgCgojIyBEYXRhIGltcG9ydAoKQWxsIHRoZSB3b3JrIEkgZG8gaW4gRHIuIEVsLVNheWVkJ3MgbGFiIG1ha2VzIHNvbWUgcHJldHR5IGhhcmQKYXNzdW1wdGlvbnMgYWJvdXQgaG93IGRhdGEgaXMgc3RvcmVkLiAgQXMgYSByZXN1bHQsIHRvIHVzZSB0aGUgZmlzc2lvbgpkYXRhIHNldCBJIHdpbGwgZG8gYSBsaXR0bGUgYml0IG9mIHNoZW5hbmlnYW5zIHRvIG1hdGNoIGl0IHRvIHRoZQpleHBlY3RlZCBmb3JtYXQuICBOb3cgdGhhdCBJIGhhdmUgcGxheWVkIGEgbGl0dGxlIHdpdGggZmlzc2lvbiwgSQp0aGluayBpdHMgZm9ybWF0IGlzIHF1aXRlIG5pY2UgYW5kIGFtIGxpa2VseSB0byBoYXZlIG15IGV4cGVyaW1lbnQKY2xhc3MgaW5zdGVhZCBiZSBhIFN1bW1hcml6ZWRFeHBlcmltZW50LgoKYGBge3IgZGF0YV9pbXBvcnR9CiMjIEV4dHJhY3QgdGhlIG1ldGEgZGF0YSBmcm9tIHRoZSBmaXNzaW9uIGRhdGFzZXQKbWV0YSA8LSBhcy5kYXRhLmZyYW1lKGZpc3Npb25AY29sRGF0YSkKIyMgTWFrZSBjb25kaXRpb25zIGFuZCBiYXRjaGVzCm1ldGEkY29uZGl0aW9uIDwtIHBhc3RlKG1ldGEkc3RyYWluLCBtZXRhJG1pbnV0ZSwgc2VwPSIuIikKbWV0YSRiYXRjaCA8LSBtZXRhJHJlcGxpY2F0ZQptZXRhJHNhbXBsZS5pZCA8LSByb3duYW1lcyhtZXRhKQojIyBHcmFiIHRoZSBjb3VudCBkYXRhCmZpc3Npb25fZGF0YSA8LSBmaXNzaW9uQGFzc2F5cyRkYXRhJGNvdW50cwojIyBUaGlzIHdpbGwgbWFrZSBhbiBleHBlcmltZW50IHN1cGVyY2xhc3MgY2FsbGVkICdleHB0JyBhbmQgaXQgY29udGFpbnMKIyMgYW4gRXhwcmVzc2lvblNldCBhbG9uZyB3aXRoIGFueSBhcmJpdHJhcnkgYWRkaXRpb25hbCBpbmZvcm1hdGlvbiBvbmUgbWlnaHQgd2FudCB0byBpbmNsdWRlLgojIyBBbG9uZyB0aGUgd2F5IGl0IHdyaXRlcyBhIFJkYXRhIGZpbGUgd2hpY2ggaXMgYnkgZGVmYXVsdCBjYWxsZWQgJ2V4cHQuUmRhdGEnCmZpc3Npb25fZXhwdCA8LSBjcmVhdGVfZXhwdChtZXRhZGF0YT1tZXRhLCBjb3VudF9kYXRhZnJhbWU9Zmlzc2lvbl9kYXRhKQpgYGAKCiMjIE5vcm1hbGl6aW5nIGFuZCBleHBsb3JpbmcgZGF0YQoKVGhlcmUgYXJlIGxvdHMgb2YgdG95cyB3ZSBoYXZlIGxlYXJuZWQgdG8gdXNlIHRvIHBsYXkgd2l0aCB3aXRoIHJhdwpkYXRhIGFuZCBleHBsb3JlIHN0dWZmIGxpa2UgYmF0Y2ggZWZmZWN0cyBvciBub24tY2Fub25pY2FsCmRpc3RyaWJ1dGlvbnMgb3Igc2tld2VkIGNvdW50cy4gIGhwZ2x0b29scyBwcm92aWRlcyBzb21lIGZ1bmN0aW9uYWxpdHkKdG8gbWFrZSB0aGlzIHByb2Nlc3MgZWFzaWVyLiAgVGhlIGdyYXBocyBzaG93biBiZWxvdyBhbmQgbWFueSBtb3JlIGFyZQpnZW5lcmF0ZWQgd2l0aCB0aGUgd3JhcHBlciAnZ3JhcGhfbWV0cmljcygpJyBidXQgdGhhdCB0YWtlcyBhd2F5IHRoZQpjaGFuY2UgdG8gZXhwbGFpbiB0aGUgZ3JhcGhzIGFzIEkgZ2VuZXJhdGUgdGhlbS4KCmBgYHtyIG5vcm1fZXhwbG9yZX0KIyMgRmlyc3QgbWFrZSBhIGJhciBwbG90IG9mIHRoZSBsaWJyYXJ5IHNpemVzIGluIHRoZSBleHBlcmltZW50LgojIyBOb3RpY2UgdGhhdCB0aGUgY29sb3JzIHdlcmUgYXV0by1jaG9zZW4gYnkgY3JlYXRlX2V4cHQoKSBhbmQgdGhleSBzaG91bGQKIyMgYmUgbWFpbnRhaW5lZCB0aHJvdWdob3V0IHRoaXMgcHJvY2VzcwpmaXNfbGlic2l6ZSA8LSBwbG90X2xpYnNpemUoZmlzc2lvbl9leHB0KQpmaXNfbGlic2l6ZSRwbG90CiMjIEhlcmUgd2Ugc2VlIHRoYXQgdGhlIHdpbGQgdHlwZSByZXBsaWNhdGUgMyBzYW1wbGUgZm9yIDE1IG1pbnV0ZXMgaGFzIGZld2VyIG5vbi16ZXJvIGdlbmVzIHRoYW4gYWxsIGl0cyBmcmllbmRzLgpmaXNfbm9uemVybyA8LSBwbG90X25vbnplcm8oZmlzc2lvbl9leHB0LCBsYWJlbHM9ImJvcmluZyIsIHRpdGxlPSJub256ZXJvIHZzLiBjcG0iKQpmaXNfbm9uemVybyRwbG90CgpmaXNfZGVuc2l0eSA8LSBwbG90X2RlbnNpdHkoZmlzc2lvbl9leHB0KQpmaXNfZGVuc2l0eSRwbG90CmZpc19kZW5zaXR5JGNvbmRpdGlvbl9zdW1tYXJ5CmZpc19kZW5zaXR5JGJhdGNoX3N1bW1hcnkKZmlzX2RlbnNpdHkkc2FtcGxlX3N1bW1hcnkKYGBgCgojIyMgQW4gaW5pdGlhbCBwY2EgcGxvdAoKSW4gbW9zdCBjYXNlcywgcmF3IGRhdGEgZG9lcyBub3QgY2x1c3RlciB2ZXJ5IHdlbGwsIGxldHMgc2VlIGlmIHRoYXQKaXMgYWxzbyB0cnVlIGZvciB0aGUgZmlzc2lvbiBleHBlcmltZW50LiBBc3N1bWluZyBpdCBkb2Vzbid0LCBsZXRzCm5vcm1hbGl6ZSB0aGUgZGF0YSB1c2luZyB0aGUgZGVmYXVsdHMgKGNwbSwgcXVhbnRpbGUsIGxvZzIpIGFuZCB0cnkKYWdhaW4uCgpgYGB7ciBwY2F9CiMjIFNvbWV0aGluZyBpbiB0aGlzIGlzIGNhdXNpbmcgYSBidWlsZCBsb29wIG9uIHRyYXZpcy4uLgojIyBVbnN1cnByaXNpbmdseSwgdGhlIHJhdyBkYXRhIGRvZXNuJ3QgY2x1c3RlciB3ZWxsIGF0IGFsbC4uLgpmaXNfcmF3cGNhIDwtIHBsb3RfcGNhKGZpc3Npb25fZXhwdCwgZXhwdF9sYWJlbHM9Zmlzc2lvbl9leHB0JGNvbmRpdGlvbiwgY2lzPU5VTEwpCmZpc19yYXdwY2EkcGxvdAojIyBTbywgbm9ybWFsaXplIHRoZSBkYXRhCm5vcm1fZXhwdCA8LSBzbShub3JtYWxpemVfZXhwdChmaXNzaW9uX2V4cHQsIHRyYW5zZm9ybT0ibG9nMiIsIG5vcm09InF1YW50IiwgY29udmVydD0iY3BtIikpCiMjIEFuZCB0cnkgdGhlIHBjYSBhZ2FpbgpmaXNfbm9ybXBjYSA8LSBwbG90X3BjYShub3JtX2V4cHQsIHBsb3RfbGFiZWxzPSJub3JtYWwiLCB0aXRsZT0ibm9ybWFsaXplZCBwY2EiLCBjaXM9TlVMTCkKZmlzX25vcm1wY2EkcGxvdAoKc3VtbWFyeShmaXNfbm9ybXBjYSkKYGBgCgpJbiB0aGUgZmluYWwgbGluZSBvZiB0aGUgcHJlY2VlZGluZyBibG9jaywgSSBwcmludGVkIGEgc3VtbWFyeSBvZiB0aGUgcmV0dXJuIGZyb20gcGxvdF9wY2EoKS4KSXQgY29udGFpbnMgdGhlIGZvbGxvd2luZyBpbmZvcm1hdGlvbjoKCiogcGNhOiAgICAgIFRoZSByZXN1bHQgZnJvbSB0aGUgZmFzdC5zdmQoKSBjYWxsLgoqIHBsb3Q6ICAgICBUaGUgZ2dwbG90MiBwY2EgcGxvdC4KKiB0YWJsZTogICAgVGhlIG1ldGFkYXRhIHVzZWQgdG8gbWFrZSB0aGUgcGNhIHBsb3QuCiogcmVzOiAgICAgIEEgdGFibGUgb2YgdGhlIHJlc2lkdWFsIHZhcmlhbmNlIGFmdGVyIGVhY2ggY29tcG9uZW50IGJ5IGNvbmRpdGlvbi9iYXRjaC4KKiB2YXJpYW5jZTogQSBudW1lcmljIGxpc3Qgb2YgdGhlICV2YXJpYW5jZSByZW1haW5pbmcgYWZ0ZXIgZWFjaCBQQy4KCldpdGggdGhhdCBpbiBtaW5kLCBsZXRzIHBlcmZvcm0gc29tZSBtb3JlIHBjYSBwbG90cyBhZnRlciBub3JtYWxpemluZyB0aGUgZGF0YSBhbmQgc2VlIGhvdyBkaWZmZXJlbnQKdGhleSBsb29rLgoKYGBge3Igbm9ybWFsaXplZF9wY2F9Cm5vcm1iYXRjaF9leHB0IDwtIHNtKG5vcm1hbGl6ZV9leHB0KGZpc3Npb25fZXhwdCwgdHJhbnNmb3JtPSJsb2cyIiwgbm9ybT0icXVhbnQiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb252ZXJ0PSJjcG0iLCBiYXRjaD0ic3ZhIikpCmZpc19ub3JtYmF0Y2hwY2EgPC0gcGxvdF9wY2Eobm9ybWJhdGNoX2V4cHQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdGl0bGU9Ik5vcm1hbGl6ZWQgUENBIHdpdGggYmF0Y2ggZWZmZWN0IGNvcnJlY3Rpb24uIiwgY2lzPU5VTEwpCmZpc19ub3JtYmF0Y2hwY2EkcGxvdAojIyBvaywgdGhhdCBjYXVzZWQgdGhlIDAsIDYwLCAxNSwgYW5kIDMwIG1pbnV0ZSBzYW1wbGVzIHRvIGNsdXN0ZXIgbmljZWx5CiMjIHRoZSAxMjAgYW5kIDE4MCBtaW51dGUgc2FtcGxlcyBhcmUgc3RpbGwgYSBiaXQgdGlnaHQKCiMjIHBjYV9pbmZvcm1hdGlvbiBwcm92aWRlcyBzb21lIG1vcmUgaW5mb3JtYXRpb24gYWJvdXQgdGhlIGNhbGwgdG8KIyMgZmFzdC5zdmQgdGhhdCB3ZW50IGludG8gbWFraW5nIHRoZSBwY2EgcGxvdApmaXNfaW5mbyA8LSBwY2FfaW5mb3JtYXRpb24obm9ybV9leHB0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgZXhwdF9mYWN0b3JzPWMoImNvbmRpdGlvbiIsImJhdGNoIiwic3RyYWluIiwibWludXRlIiksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBudW1fY29tcG9uZW50cz02KQojIyBUaGUgcl4yIHRhYmxlIHNob3dzIHRoYXQgcXVpdGUgYSBsb3Qgb2YgdGhlIHZhcmlhbmNlIGluIHRoZSBkYXRhIGlzIGV4cGxhaW5lZCBieSBjb25kaXRpb24KaGVhZChmaXNfaW5mbyRyc3F1YXJlZF90YWJsZSkKIyMgV2UgY2FuIGxvb2sgYXQgdGhlIGNvcnJlbGF0aW9uIGJldHdlZW4gdGhlIHByaW5jaXBsZSBjb21wb25lbnRzIGFuZCB0aGUgZmFjdG9ycyBpbiB0aGUgZXhwZXJpbWVudAojIyBpbiB0aGlzIGNhc2UgbG9va2luZyBhdCBjb25kaXRpb24vYmF0Y2ggdnMgdGhlIGZpcnN0IDQgY29tcG9uZW50cy4KZmlzX2luZm8kcGNhX2NvcgojIyBBbmQgcC12YWx1ZXMgdG8gbGVuZCBzb21lIGNyZWRlbmNlKG9yIG5vdCB0byB0aG9zZSBhc3NlcnRpb25zKQpmaXNfaW5mbyRhbm92YV9wCgojIyBUcnkgYWdhaW4gd2l0aCBiYXRjaCByZW1vdmVkIGRhdGEKYmF0Y2hub3JtX2V4cHQgPC0gc20obm9ybWFsaXplX2V4cHQoZmlzc2lvbl9leHB0LCBiYXRjaD0ibGltbWEiLCBub3JtPSJxdWFudCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIpKQpmaXNfYmF0Y2hub3JtcGNhIDwtIHBsb3RfcGNhKGJhdGNobm9ybV9leHB0LCBwbG90X3RpdGxlPSJsaW1tYSBjb3JyZWN0ZWQgcGNhIiwgY2lzPU5VTEwpCmZpc19iYXRjaG5vcm1wY2EkcGxvdAp0ZXN0X3BjYSA8LSBwY2FfaW5mb3JtYXRpb24oYmF0Y2hub3JtX2V4cHQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBleHB0X2ZhY3RvcnM9YygiY29uZGl0aW9uIiwiYmF0Y2giLCJzdHJhaW4iLCJtaW51dGUiKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIG51bV9jb21wb25lbnRzPTYpCmBgYAoKSW50ZXJlc3RpbmcsIHRoZSBiYXRjaCBub3JtYWxpemVkIHBjYSBwbG90IGxvb2tzIG11Y2ggdGhlIHNhbWUgYXMgdGhlCm5vcm1hbGl6ZWQuIFRoZSB2YXJpYW5jZXMgYXJlIGluIGZhY3QgcHJldHR5IG11Y2ggdGhlIGV4YWN0IHNhbWUuLi4KCiMjIExvb2sgYXQgdGhlIGRhdGEgZGlzdHJpYnV0aW9ucwoKV2UgaGF2ZSBzb21lIHRvb2xzIHdoaWNoIHByb3ZpZGUgdmlzdWFsaXphdGlvbnMgb2YgdGhlIGRpc3RyaWJ1dGlvbiBvZgp0aGUgZGF0YToKCmBgYHtyIGRpc3RyaWJ1dGlvbnN9CmZpc3Npb25fYm94cGxvdCA8LSBzbShwbG90X2JveHBsb3QoZmlzc2lvbl9leHB0KSkKZmlzc2lvbl9ib3hwbG90CnNmX2V4cHQgPC0gc20obm9ybWFsaXplX2V4cHQoZmlzc2lvbl9leHB0LCBub3JtPSJzZiIpKQpmaXNzaW9uX2JveHBsb3QgPC0gc20ocGxvdF9ib3hwbG90KHNmX2V4cHQpKQpmaXNzaW9uX2JveHBsb3QKdG1fZXhwdCA8LSBzbShub3JtYWxpemVfZXhwdChmaXNzaW9uX2V4cHQsIG5vcm09InRtbSIpKQpmaXNzaW9uX2JveHBsb3QgPC0gc20ocGxvdF9ib3hwbG90KHRtX2V4cHQpKQpmaXNzaW9uX2JveHBsb3QKcmxlX2V4cHQgPC0gc20obm9ybWFsaXplX2V4cHQoZmlzc2lvbl9leHB0LCBub3JtPSJybGUiKSkKZmlzc2lvbl9ib3hwbG90IDwtIHNtKHBsb3RfYm94cGxvdChybGVfZXhwdCkpCmZpc3Npb25fYm94cGxvdAp1cF9leHB0IDwtIHNtKG5vcm1hbGl6ZV9leHB0KGZpc3Npb25fZXhwdCwgbm9ybT0idXBwZXJxdWFydGlsZSIpKQpmaXNzaW9uX2JveHBsb3QgPC0gc20ocGxvdF9ib3hwbG90KHVwX2V4cHQpKQpmaXNzaW9uX2JveHBsb3QKCmZpc3Npb25fZGVuc2l0eSA8LSBwbG90X2RlbnNpdHkobm9ybV9leHB0KQpmaXNzaW9uX2RlbnNpdHkkcGxvdApmaXNzaW9uX2RlbnNpdHkgPC0gcGxvdF9kZW5zaXR5KHNmX2V4cHQpCmZpc3Npb25fZGVuc2l0eSRwbG90CmZpc3Npb25fZGVuc2l0eSA8LSBwbG90X2RlbnNpdHkodG1fZXhwdCkKZmlzc2lvbl9kZW5zaXR5JHBsb3QKCmNvbXBhcmVfMTIgPC0gcGxvdF9zaW5nbGVfcXEoZmlzc2lvbl9leHB0LCB4PTEsIHk9MikKY29tcGFyZV8xMiRsb2cKYGBgCgojIyBTZWUgaG93IHRoZXkgY2x1c3RlcgoKT2ssIHNvIHdlIGNhbiBmdXJ0aGVyIGNoZWNrIG91dCBob3cgdGhlIGRhdGEgY2x1c3RlciB3aXRoIHJlc3BlY3QgdG8Kb25lIGFub3RoZXIuLi4KCmBgYHtyIGNsdXN0ZXJpbmd9CmZpc3Npb25fY29yIDwtIHBsb3RfY29yaGVhdChub3JtX2V4cHQpCmZpc3Npb25fY29yJHBsb3QKZmlzc2lvbl9jb3IgPC0gcGxvdF9jb3JoZWF0KGJhdGNobm9ybV9leHB0KQpmaXNzaW9uX2NvciRwbG90CmZpc3Npb25fZGlzIDwtIHBsb3RfZGlzaGVhdChub3JtX2V4cHQpCmZpc3Npb25fZGlzJHBsb3QKZmlzc2lvbl9kaXMgPC0gcGxvdF9kaXNoZWF0KGJhdGNobm9ybV9leHB0KQpmaXNzaW9uX2RpcyRwbG90CmBgYAoKIyB2YXJpYW5jZVBhcnRpdGlvbgoKQSBuZXcgdG95IEkgaGF2ZSBiZWVuIGludHJvZHVjZWQgdG8gaXMgdmFyaWFuY2VQYXJ0aXRpb24sIGxldCB1cyB0cnkgdGhhdCB3aXRoIHRoaXMgZGF0YS4KCmBgYHtyIHZhcmlhbmNlUGFydGl0aW9ufQp0ZXN0X3ZhcnBhcnQgPC0gdmFycGFydChmaXNzaW9uX2V4cHQsIHByZWRpY3Rvcj1OVUxMLCBmYWN0b3JzPWMoImNvbmRpdGlvbiIsImJhdGNoIikpCnRlc3RfdmFycGFydCRwZXJjZW50X3Bsb3QKdGVzdF92YXJwYXJ0JHBhcnRpdGlvbl9wbG90CgojIyBIZXJlLCBsZXQgdXMgdGVzdCB0aGUgdmFyaWFuY2UgY29udHJpYnV0ZWQgYnkgc3RyYWluLCB0aW1lLCBhbmQgcmVwbGljYXRlLgp0ZXN0X3ZhcnBhcnQgPC0gdmFycGFydChmaXNzaW9uX2V4cHQsIHByZWRpY3Rvcj1OVUxMLCBmYWN0b3JzPWMoImNvbmRpdGlvbiIsICJzdHJhaW4iLCAibWludXRlIiwgInJlcGxpY2F0ZSIpKQp0ZXN0X3ZhcnBhcnQkcGVyY2VudF9wbG90CnRlc3RfdmFycGFydCRwYXJ0aXRpb25fcGxvdApgYGAKCllBWSEKCltpbmRleC5odG1sXShpbmRleC5odG1sKQoKYGBge3Igc3lzaW5mbywgcmVzdWx0cz0nYXNpcyd9CnBhbmRlcjo6cGFuZGVyKHNlc3Npb25JbmZvKCkpCmBgYAo=